Skip to content

stream: cut per-chunk overhead in web streams pipelines - #65437

Open
anonrig wants to merge 3 commits into
nodejs:mainfrom
anonrig:cursor/webstreams-pipeline-perf-93a7
Open

stream: cut per-chunk overhead in web streams pipelines#65437
anonrig wants to merge 3 commits into
nodejs:mainfrom
anonrig:cursor/webstreams-pipeline-perf-93a7

Conversation

@anonrig

@anonrig anonrig commented Aug 20, 2026

Copy link
Copy Markdown
Member

Reduces per-chunk overhead across common web streams pipelines (fetch body decompression/upload, TextDecoderStream/TextEncoderStream transcoding), targeting workloads that move large payloads in small (e.g. 4 KiB) chunks.

Changes

stream: process CompressionStream chunks without threadpool round trips

CompressionStream/DecompressionStream wrapped a zlib stream.Duplex in the web streams adapters, so every written chunk was dispatched to the threadpool and its completion observed on a later event loop turn. For a 64 MiB body written in 4 KiB chunks that is 16384 threadpool round trips plus the Transform and adapter machinery around them. The classes are now built on a TransformStream that drives the raw zlib/brotli handle synchronously, mirroring the write loop of the zlib streams (same handle setup, same error mapping, same trailing-garbage rejection). Inputs larger than 64 KiB are processed in slices with an event-loop turn in between, so a huge chunk cannot block the loop for its full duration. Output is emitted in up to 64 KiB chunks, either as zero-copy views or as right-sized copies so tiny chunks do not retain large buffers.

stream: fast-path TextEncoderStream chunk encoding

The encode-and-enqueue algorithm was a literal transcription of the spec's per-code-unit loop: it extracted a single-character string, called charCodeAt(), and appended to an accumulator string for every code unit of every chunk (~67 million temporary strings for a 64 MiB payload). The loop's only observable effects are the surrogate hand-off at chunk boundaries and U+FFFD replacement of unpaired surrogates, which TextEncoder.encode() already performs; the transform now handles the chunk boundary explicitly and encodes the rest of the chunk with a single encode() call.

stream: remove unused sync-error destroy plumbing from adapters

The kValidateChunk/kDestroyOnSyncError hooks existed only for the previous Duplex-based compression implementation.

Benchmarks

64 MiB of log-like text, 4 KiB chunks, best of 5, idle Linux x64. Baseline is v27.0.0-nightly20260819 (current main); "this PR" is a from-source build of this branch.

End-to-end pipelines

Pipeline main this PR speedup
Download (fetch → gunzip → decode → for await) 154 ms · 417 MB/s 96 ms · 668 MB/s 1.6x
Upload (fs → gzip → fetch POST) 1154 ms · 56 MB/s 750 ms · 85 MB/s 1.5x
Transcode (fs → decode → encode → fs) 771 ms · 83 MB/s 204 ms · 315 MB/s 3.8x
Subprocess (fetchcatfor await) 67–81 ms · ~800–950 MB/s 77–83 ms · ~780–830 MB/s ~1.0x (already at the pipe/syscall floor)

Isolated stages (64 MiB through just the stage, 4 KiB chunks)

Stage main this PR speedup
TextEncoderStream 612 ms · 105 MB/s 44 ms · 1464 MB/s 13.9x
CompressionStream('gzip') 306 ms · 209 MB/s 136 ms · 471 MB/s 2.3x
DecompressionStream('gzip') 79 ms · 807 MB/s 36 ms · 1796 MB/s 2.2x

End-to-end wall time is capped by physical floors on this machine (gzipSync of this payload is ~620 ms of raw CPU; gunzipSync is ~139 ms; UTF-8 decode+encode of 64 MiB is ~84 ms). After this PR the remaining non-codec overhead on upload is ~130 ms (~8 µs/chunk) and on download is well under the gunzip floor. A literal 10x on wall time is not available for upload/download/subprocess; the one place a full 10x+ existed was the encoder stage.

Adds benchmark/webstreams/compression.js and benchmark/webstreams/encoding.js.

Behavior notes

  • Compression/decompression now runs on the JS thread. For small chunks this strictly removes latency (the deflate of a 4 KiB chunk costs far less than a threadpool dispatch); for oversized chunks the input is sliced with event-loop yields to bound blocking.
  • Decompressed/compressed output chunks may now be emitted as Uint8Array views over a larger ArrayBuffer (when at least half the buffer is filled) rather than always exact-sized copies. Those views never share memory with one another: the backing buffer is retired after a view is emitted.
  • Node.js-specific behaviors are preserved and covered by tests: string chunks are accepted, null rejects with ERR_STREAM_NULL_VALUES, non-BufferSource chunks reject with ERR_INVALID_ARG_TYPE, trailing garbage rejects with ERR_TRAILING_JUNK_AFTER_STREAM_END (all TypeErrors), and 'brotli' remains supported.

Testing

Local out/Release/node (this branch): 30 targeted tests, all passing, including test-whatwg-webstreams-compression, test-webstreams-compression-bad-chunks, test-webstreams-decompression-reject-trailing, test-webstreams-compression-buffer-source, test-compression-decompression-stream, test-zlib-type-error, test-whatwg-webstreams-encoding, the Readable.toWeb / adapters tests, and WPT compression, encoding, and streams.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. web streams labels Aug 20, 2026
@anonrig
anonrig marked this pull request as ready for review August 20, 2026 16:29
@cursor
cursor Bot force-pushed the cursor/webstreams-pipeline-perf-93a7 branch from 796edc7 to 8019292 Compare August 20, 2026 16:30
Rewrite CompressionStream and DecompressionStream on top of a
TransformStream that drives the zlib (or brotli) handle synchronously,
instead of wrapping a zlib stream.Duplex in the web streams adapters.

The previous implementation dispatched every written chunk to the
threadpool and waited for the event loop to observe its completion,
which dominates the cost of streaming small chunks: a 64 MiB body
written in 4 KiB chunks paid for 16384 threadpool round trips plus the
Transform and adapter machinery around them. Processing the chunks
inline removes that latency entirely while performing the same work.
Inputs larger than 64 KiB are processed in slices with a turn of the
event loop in between so that huge chunks cannot block the loop for
their full duration.

Output is emitted in up to 64 KiB chunks, either as zero-copy views or
as right-sized copies (so small outputs do not retain large buffers),
which also reduces the per-chunk overhead imposed on the rest of the
pipeline downstream.

Assisted-by: Cursor
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
The encode-and-enqueue algorithm was implemented as a literal
transcription of the spec's per-code-unit loop: it extracted a
single-character string, called charCodeAt(), and appended to an
accumulator string for every code unit of every chunk, allocating
millions of temporary strings for large payloads.

The only observable effects of that loop are that a high surrogate at
the end of a chunk is held back to pair with a low surrogate starting
the next chunk, and that unpaired surrogates encode as U+FFFD, which
TextEncoder already does. Handle the chunk boundary explicitly and
encode the rest of the chunk with a single encode() call.

Assisted-by: Cursor
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
@cursor
cursor Bot force-pushed the cursor/webstreams-pipeline-perf-93a7 branch from 8019292 to 9d691c1 Compare August 20, 2026 16:39

@jasnell jasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI agents cannot used Signed-off-by

@anonrig

anonrig commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

AI agents cannot used Signed-off-by

Sorry for the frustration. My AI keeps making the same mistake. Fixing it now.

@cursor
cursor Bot force-pushed the cursor/webstreams-pipeline-perf-93a7 branch from 9d691c1 to eea0fe9 Compare August 20, 2026 16:42
@jasnell
jasnell dismissed their stale review August 20, 2026 16:44

Resolved

@cursor
cursor Bot force-pushed the cursor/webstreams-pipeline-perf-93a7 branch from eea0fe9 to 936186f Compare August 20, 2026 16:45
The kValidateChunk and kDestroyOnSyncError hooks existed only for the
previous stream.Duplex-based CompressionStream implementation, which no
longer uses the adapters.

Assisted-by: Cursor
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
@cursor
cursor Bot force-pushed the cursor/webstreams-pipeline-perf-93a7 branch from 936186f to e3f6bd2 Compare August 20, 2026 17:11
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.22807% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.11%. Comparing base (f83e7df) to head (e3f6bd2).
⚠️ Report is 118 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/webstreams/compression.js 90.38% 35 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65437      +/-   ##
==========================================
- Coverage   90.31%   90.11%   -0.20%     
==========================================
  Files         751      752       +1     
  Lines      249956   252618    +2662     
  Branches    47204    47503     +299     
==========================================
+ Hits       225745   227649    +1904     
- Misses      15612    16252     +640     
- Partials     8599     8717     +118     
Files with missing lines Coverage Δ
lib/internal/webstreams/adapters.js 85.08% <100.00%> (-1.62%) ⬇️
lib/internal/webstreams/encoding.js 100.00% <100.00%> (ø)
lib/internal/webstreams/compression.js 93.39% <90.38%> (-6.61%) ⬇️

... and 106 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. web streams

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants